50 Gb/s Ethernet using serializer/deserializer lanes

ABSTRACT

Systems, devices, and methods of implementing 50 Gb/s Ethernet using serializer/deserializer lanes are disclosed. One such device includes circuitry operable to provide a media access control (MAC) interface. The MAC interface is associated with a port having a 50 Gb/s link rate. The device also includes circuitry operable to generate Ethernet frames from data received at the MAC interface and circuitry operable to distribute the Ethernet frames across a group of serial/deserializer (SERDES) lanes associated with the port, the group having size N. The device also includes circuitry operable to transmit the distributed Ethernet frames on each of the SERDES lanes at a 50/N Gb/s rate.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 13/752,756, filedJan. 29, 2013, the entire contents of which is incorporated herein byreference. U.S. Ser. No. 13/752,756 claims the benefit of priority under119(e) of U.S. Provisional Application No. 61/671,966, entitled “50 Gb/sEthernet Technology for LAN Applications,” filed Jul. 16, 2012, and alsoclaims the benefit of U.S. Provisional Application No. 61/752,070,entitled “50 Gb/s Ethernet Using Serializer/Deserializer Lanes,” filedJan. 14, 2013.

TECHNICAL FIELD

The present disclosure generally relates to communications systems, andmore specifically, to Ethernet networks.

BACKGROUND

Server computer networks suitable for enterprise or cloud computingtypically utilize a scalable server infrastructure to host applications.A network fabric comprised of switches is often used since it scales tolarger node counts without impacting the cost per server. Today's bladeservers often use one of the high speed varieties of the IEEE 802.3standard for switching within the blade server.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the disclosure can be better understood with referenceto the following drawings. The components in the drawings are notnecessarily to scale, emphasis instead being placed upon clearlyillustrating the principles of the present disclosure. Moreover, in thedrawings, like reference numerals designate corresponding partsthroughout the several views.

FIG. 1 is a block diagram illustrating a blade server system accordingto some embodiments disclosed herein.

FIGS. 2A-C are block diagrams illustrating various embodiments of theLAN On Motherboard (LOM) of FIG. 1, differing in the number of lanesused per link.

FIG. 3 illustrates an OUI tagged formatted Next Page as used duringauto-negotiation by some embodiments of the LAN On Motherboard (LOM) ofFIG. 1.

FIG. 4 illustrates another OUI tagged formatted Next Page as used duringauto-negotiation by some embodiments of the LAN On Motherboard (LOM) ofFIG. 1.

FIG. 5 is a block diagram illustrating a dual-port embodiment of the LANOn Motherboard (LOM) of FIG. 1.

FIG. 6 is a block diagram illustrating some of the functional blockswithin the LAN On Motherboard (LOM) of FIG. 1.

FIGS. 7A-C are block diagrams illustrating various embodiments of theLAN On Motherboard (LOM) of FIG. 1, showing striping across variousnumbers of Physical Coding Sublayer (PCS) lanes.

FIG. 8 is a flowchart illustrating transmit operation of the LAN OnMotherboard (LOM) of FIG. 1, according to some embodiments disclosedherein.

FIG. 9 is a flowchart illustrating receive operation of the LAN OnMotherboard (LOM) of FIG. 1, according to some embodiments disclosedherein.

DETAILED DESCRIPTION

According to the inventive techniques disclosed herein, an Ethernetcontroller distributes Ethernet frames across multipleserializer/deserializer (SERDES) lanes for conveyance across a bladeserver midplane in a manner that achieves a data rate of up to 50 Gb/s.Embodiments of the Ethernet controller can be implemented as a separatenetwork interface controller (NIC) and physical layer transceiver (PHY),or can integrate the Media Access Control (MAC) and PHY functions onto asingle chip.

One embodiment includes circuitry operable to provide a MAC interfaceassociated with a port having a 50 Gb/s rate. This embodiment alsoincludes circuitry operable to generate Ethernet frames from datareceived at the MAC interface, and to distribute the Ethernet framesover a plurality of serializer/deserializer (SERDES) lanes. Theplurality has size N and each SERDES lane is operable at a 50/N Gb/srate. This embodiment also includes circuitry operable to transmit thedistributed Ethernet frames over the N SERDES lanes.

Another embodiment involves a network interface controller (NIC). TheNIC includes host interface logic and physical layer (PHY) logic. Thehost interface logic is operable provide a MAC interface associated witha port having a 50 Gb/s rate and to generate Ethernet frames from datareceived at the MAC interface. The PHY logic is operable to distributethe Ethernet frames over a plurality of serializer/deserializer (SERDES)lanes. The plurality has size N and each SERDES lane is operable at a50/N Gb/s rate. This embodiment also includes circuitry operable totransmit the distributed Ethernet frames over the N SERDES lanes.

Yet another embodiment involves a dual port NIC. The dual port NIC isoperable to provide a pair of 50 Gb/s Media Access Control (MAC) ports.The dual port NIC is further operable to receive first Ethernet framesat the first 50 Gb/s MAC Port and to and transmit the first Ethernetframes over a first plurality of serializer/deserializer (SERDES) lanesassociated with the first 50 Gb/s MAC Port. The first plurality has sizeN and each of the first SERDES lanes is operable to transmit at a 50/NGb/s rate. The dual port NIC is further operable to receive secondEthernet frames over a second plurality of serializer/deserializer(SERDES) lanes associated with the second 50 Gb/s MAC port. The secondplurality has size N and each of the second SERDES lanes is operable totransmit at a 50/N Gb/s rate. The dual port NIC is further operable totransmit the second received Ethernet frames over the second 50 Gb/s MACport.

Still another embodiment involves device including a plurality ofserializer/deserializers (SERDESes) and a network interface controller(NIC). Each SERDES implements a corresponding SERDES lane. The pluralityof SERDESes has size N. Each SERDES lane is operable to transmit at a50/N Gb/s rate. The NIC includes host interface logic and physical layer(PHY) logic. The host interface logic is operable to provide a 50 Gb/sMedia Access Control (MAC) port and to receive Ethernet frames at 50Gb/s at the MAC port. The physical interface logic is operable totransmit the Ethernet frames over the SERDES lanes.

Yet another embodiment involves a network interface controller (NIC).The NIC is operable to receive Ethernet frames from a media accesscontrol (MAC) layer at a 50 Gb/s rate, and transmit the Ethernet framesover one or more serializer/deserializer (SERDES) lanes. Each of theSERDES lanes is operable at a 50/N Gb/s rate, where N is the number ofthe one or more SERDES lanes.

Having summarized various aspects of the present disclosure, referencewill now be made in detail to the description of the disclosure asillustrated in the drawings. While the disclosure will be described inconnection with these drawings, there is no intent to limit it to theembodiment or embodiments disclosed herein. On the contrary, the intentis to cover all alternatives, modifications, and equivalents includedwithin the spirit and scope of the disclosure as defined by the appendedclaims.

FIG. 1 is a block diagram of a blade server system including a 50 Gb/sEthernet controller, according to some embodiments described herein. Theblade server system 100 includes various cards 110 in data communicationvia a midplane 120. The midplane 120 is a printed circuit board (PCB)that mounts on a blade server chassis and provides power distribution,fabric connectivity, and system management infrastructure to the bladeserver system 100. The midplane 120 is generally formed of multiplelayers of dielectric substrates with conductive paths, referred toherein as traces, formed on one or more of the dielectric layers. Cards110 plug into the midplane 120 by way of connectors (not shown) on boththe midplane 120, and the traces on the midplane 120 provides electricalcoupling therebetween. As FIG. 1 is a block diagram rather than anactual illustration, traces are represented schematically and are notmeant to show physical location.

In the example blade server system 100, the cards 110 connected by themidplane 120 include one or more server blades 130 and one or morenetwork switches 140. Through the midplane 120, the server blades 130are connected to the network switches 140, which in turn allows thevarious server blades 130 to communicate with each other and withcomponents, devices, and systems external to the blade server system100. The blade server system 100 may reside, for example, in a datacenter that is in communication with the Internet. In the embodiment ofFIG. 1, the network switches 140 are used in a redundant configuration,with network switch 140-1 active and network switch 140-2 on standby.

A server blade 130 includes one or more host processors 150, a memorycontroller and I/O hub 160, and a 50 Gb/s Local Area Network (LAN) OnMotherboard 170. A host processor 150 communicates data to the LAN OnMotherboard (LOM) 170, which encapsulates the data into Ethernet framesand transmits the Ethernet frames over the midplane 120 to the networkswitch 140. In this manner, Ethernet frames are communicated from thehost processor 150 to the LOM 170, to the midplane 120, to the networkswitch 140. From the network switch 140, the Ethernet frames may beconveyed to another network switch 140 in the blade server system 100,to another blade server system 100 in the data center, or to theInternet. In the reverse direction, the LOM 170 receives Ethernet framesfrom the network switch 140 over the midplane 120 and provides the dataencapsulated in the Ethernet frame to the host processor 150.

More specifically, an LOM 170 includes one or more ports 180, where eachport 180 connects to a specific group of traces on the midplane 120.Each of the trace groups then connects to a port 190 of a network switch140. In this manner, each LOM port 180 is electrically coupled to one ofthe switch ports 190 by a group of traces. A group of traces maycomprise, for example, two pairs of differential signals: a transmitpositive signal; a transmit negative signal; a receive positive signal;and a receive negative signal.

The group of traces associated with a single LOM port 180 is referred toherein as a link 195. A link 195 thus represents a single communicationtransport path between an LOM port 180 and a switch port 190. The LOMs170 in FIG. 1 have two ports. Thus, the first LOM 170-1 includes link195-1-A and link 195-1-B while the second LOM 170-2 includes link195-2-A and link 195-2-B. The embodiment of FIG. 1 includes redundancyfeatures, whereby link 195-1-A and link 195-2-A are active while link195-1-B and link 195-2-B are on standby. In other embodiments withoutredundant switches, both switches 140-1 and 140-2 are active, with allfour links 195-1-A, 195-1-B, 195-2-A and 195-2B being active as well. Innon-redundant embodiments, the ports 190 on both switches 140-1 and140-2 can handle 50 Gb/s Ethernet traffic.

While represented as a single line in FIG. 1, various embodiments of theLAN On Motherboard 170 implement a link 195 as a group ofserializer/deserializer (SERDES) lanes. As explained in more detailbelow, an LOM 170 associates lanes with links an inventive manner toachieve data rate of 50 Gb/s per port across the midplane 120.

It will be understood that the specific number of components illustratedin FIG. 1 is not limiting. Although the LOM 170 shown in FIG. 1 includestwo ports, other embodiments may include more than two ports, whilestill other embodiments include a single port. Similarly, although theserver blade 130 is shown with two host processors 150 in FIG. 1, othernumbers of host processors 150 are also contemplated. Also, other typesof cards 110 may be present in a blade server system 100. Havingdiscussed the role of the LAN On Motherboard 170 in the overall bladeserver system 100, the data communication function of the LOM 170 willnow be described in more detail.

The SERDES lane distribution techniques disclosed herein are describedin the context of an Ethernet controller implemented as part of a LAN OnMotherboard integrated circuit. However, the same techniques can beemployed in an Ethernet network switch MAC port operating at 50 Gb/s, ormore generically, can be employed in any Ethernet controller operatingat 50 Gb/s. Using the SERDES lane distribution techniques disclosedherein, Ethernet controllers are thus operable to transmit and receiveEthernet frames over one or more SERDES lanes. Each of the SERDES lanesis operable at a 50/N Gb/s rate, where N is the number of the one ormore SERDES lanes.

FIGS. 2A-C are block diagrams showing various configurations orembodiments of the LAN On Motherboard 170, and the number of lanes usedper link for each. The LOM 170 receives data from the host processor 150over a host data path 210 and also sends data to the host processor 150over the host data path 210. The host data path 210 may be implementedas a bus, for example, PCI-Express. Viewed logically, the data is sentto, and received from, a particular port 180 of the LOM 170. That is,the host processor 150 sends data to, and receives data from, aparticular port 180 of the LOM 170 rather than to and from the LOM 170as a whole. Because that port 180 is associated with a Media AccessControl (MAC) address, a port 180 may be referred to herein as a MACport 180. A single MAC port 180 is shown in FIGS. 2A-C for ease ofexplanation, but multi-port LOMs will be discussed later.

As discussed earlier in connection with FIG. 1, the LAN On Motherboard170 supports a 50 Gb/s link rate. This is achieved by distributingoutgoing traffic among multiple serializer/deserializers (SERDES) lanes,and in the reverse direction, aggregating traffic from multiple SERDESlanes into a single LOM port 180. While particular numbers of SERDESlanes will be discussed in connection with FIGS. 2A-C, in the generalcase each SERDES lane operates at rate up to 50/N, where N is the numberof SERDES lanes 220 implemented by the LOM 170.

In the embodiment of FIG. 2A, 50 Gb/s is achieved by communicating theEthernet frames generated by the LOM 170-A over two lanes 220-1 and220-2 provided by corresponding serializer/deserializers (SERDESes),where each of the SERDES lanes 220-1 and 220-2 operates at 25 Gb/s. TheLOM 170 bonds the two lanes (220-1 and 220-2) together using amulti-lane distribution technique to create a single logical MAC port180 operating at 50 Gb/s. This configuration permits blade serverchasses that are midplane-resource limited to overcome the link speedlimitation of Ethernet ports operating at 10 Gb/s or 20 Gb/s. In someembodiments, a 25 Gb/s SERDES lane 220 is implemented by using one ofthe four 25 Gb/s SERDES lanes of a 100 Gb/s physical layer transceiver(PHY), where the other lanes of the 100 Gb/s PHY may be used by anotherLOM 170-A, or by another MAC port of the same LOM 170-A. In such animplementation, the LOM 170-A splits the 100 Gb/s PHY into four physicaland logical ports, each operating at up to 50 Gb/s.

FIG. 2B illustrates another embodiment of the LAN On Motherboard 170,one which utilizes four SERDES lanes. In this embodiment, the LOM 170communicates the Ethernet frames generated by the LOM 170-B over lanes220-1, 220-2, 220-3, and 220-4 provided by corresponding SERDESes, whereeach of the SERDES lanes 220-1, 220-2, 220-3, and 220-4 operates at 12.5Gb/s. The LOM 170 bonds four lanes (220-1 . . . 4) together using amulti-lane distribution technique to create a single logical MAC port180 operating at 50 Gb/s. This configuration permits blade serverchasses having four SERDES lanes per port to send data at a 20% fasterrate than if the same four lanes were running at 10 Gb/s speed, as witha typical IEEE 40 G port. In some embodiments, a 12.5 Gb/s SERDES lane220 is implemented by increasing the data rate of a 10 Gb/s SERDES laneby 25%.

FIG. 2C illustrates another embodiment of the LAN On Motherboard 170,one which utilizes one SERDES lane. In this embodiment, the LOM 170communicates the Ethernet frames generated by the LOM 170-B over asingle lane 220-1 provided by a single SERDES operating at 50 Gb/s. Insome embodiments, a 50 Gb/s SERDES lane 220 is implemented by using oneof the four 25 Gb/s SERDES lanes of a 100 Gb/s physical layertransceiver (PHY), where the other lanes of the 100 Gb/s PHY may be usedby another LOM 170-A, or by another MAC port of the same LOM 170-A. Insuch an implementation, the LOM 170-A splits the 100 Gb/s PHY into twophysical and logical ports, each operating at up to 50 Gb/s, for aneconomical dual-port 50 G implementation that can serve networking portredundancy needs.

In each of the embodiments of FIGS. 2A-C, the LAN On Motherboard 170represents the 50 Gb/s interface as a MAC port 180. Some embodiments ofthe LOM 170 can auto-negotiate link speed through extended capabilitiesregister(s) implemented by the MAC port 180. In some embodiments, the 50Gb/s rate is implemented using proprietary bit locations in the linkcapabilities register(s). The link can also be forced to run at 50 Gb/swithout advertising an auto-negotiation capability.

In some embodiments, the auto-negotiation advertising is conducted onlane 0 of the SERDES and is in accordance with IEEE 802.3anspecifications. Distinction is made in the auto-negotiation bits thathelp the far end link partner to identify whether 50 Gb/s is beingadvertised for two-SERDES-lanes-mode running at 25 Gb/s each, orfour-SERDES-lanes-mode running at 12.5 Gb/s each. In some embodiments,the 50 G auto-negotiation integrates into the Physical Code Sublayer(PCS) defined by IEEE 802.3 for standardized speeds (e.g., 10 G, 40 G)and leverages some of the reserved fields. This allows the LAN OnMotherboard 170 to advertise 50 G capability along with other supportedspeeds, which permits the link to operate at any of the advertisedspeeds under IEEE 802.3 clause 73 auto-negotiation. A link partner thatdoes not recognize 50 G speed indication could then choose to link atone of the other speeds that is supported.

The auto-negotiation base pages defined by 802.3an may be exchangedbetween the two ends of the backplane channel, with the exchange takingplace on physical lane 0. After the exchange of the base page, the linkpartners may exchange an Organization Unique Identifier (OUI) taggedformatted Next Page (using message code #5) and then an OUI taggedunformatted Next Page with an extended technology abilities field, asdetailed below. The link operating speed is determined by the highestcommon denominator advertised by the link partners, and resolvedaccording to the priority table shown below as Table 1:

TABLE 1 Priority Technology Capability 1 100GBASE-CR10 100 Gb/s, 10 lane2 50GBASE-CR1  50 Gb/s, 1 lane, copper cable 3 50GBASE-KR1  50 Gb/s, 1lane, backplane traces 4 50GBASE-CR2  50 Gb/s, 2 lanes, copper cable 550GBASE-KR2  50 Gb/s, 2 lanes, backplane traces 6 50GBASE-CR4  50 Gb/s,4 lanes, copper cable 7 50GBASE-KR4  50 Gb/s, 4 lanes, backplane traces8 40GBASE-CR4  40 Gb/s, 4 lanes, copper cable 9 40GBASE-KCR4  40 Gb/s, 4lanes, backplane traces 10 20GBASE-CR2  20 Gb/s, 2 lanes, copper cable11 20G-MLD2  20 Gb/s, 2 lanes, backplane traces 12 10GBASE-KR2  10 Gb/s,1 lane, backplane traces 13 10GBASE-KX4  10 Gb/s, 4 lanes, backplanetraces 14 10GBASE-KX  1 Gb/s, 1 lane, backplane traces

FIG. 3 illustrates an OUI tagged formatted Next Page used duringauto-negotiation, according to some embodiments of LAN On Motherboard170. Next Page 300 includes an OUI field 310 and a message code field320. In this example, the value in the OUI field 310 is avendor-specific value of 000AF7(h) and the value in the message codefield 320 is 5.

FIG. 4 illustrates another OUI tagged formatted Next Page used duringauto-negotiation, according to some embodiments of LAN On Motherboard170. Next Page 400 includes an OUI field 410, a message code field 420,and an extended technologies field 430. In this example, the value inthe OUI field 410 is a vendor-specific value of 000AF7(h), the value inthe message code field 420 is 3 to indicate that the page describesextended technology abilities, and the value in the extended technologycapabilities field is set to indicate 50 Gb/s operation.

In addition to the SERDES lane bonding described above, a MAC port 180on the LAN On Motherboard 170 can be bonded with 50 Gb/s ports or otherspeed ports for all Ethernet traffic using any of the commonly usedbonding protocols used to team Ethernet ports (e.g., IEEE 802.1AX LinkAggregation).

FIG. 5 is a block diagram illustrating a dual-port embodiment of the LANOn Motherboard 170 of FIG. 1. While the embodiments discussed inconnection with FIGS. 2A-C provided one MAC port 180, the embodiment ofFIG. 3 includes two MAC ports 180-1 and 180-2. Each of the MAC ports180-1 and 180-2 is associated with a corresponding host data path 210-1and 210-2. MAC port 180-1 is associated with SERDES lanes 220-1 and220-2, each operating at 25 Gb/s. MAC port 180-2 is associated withSERDES lanes 220-3 and 220-4, each operating at 25 Gb/s. That is, datafrom host data path 210-1 becomes Ethernet frames which are transmittedon SERDES lanes 220-1 and 220-2 while data from host data path 210-2becomes Ethernet frames which are transmitted on SERDES lanes 220-3 and220-4. In the reverse direction, Ethernet frames received on SERDESlanes 220-1 and 220-2 are provided to the host processor 150 over hostdata path 210-1 while Ethernet frames received on SERDES lanes 220-3 and220-4 are provided to the host processor 150 over host data path 210-2.In some embodiments, the SERDES lanes 220-1 . . . 4 are implemented byusing the four 25 Gb/s SERDES lanes of a 100 Gb/s physical layertransceiver (PHY). Having described the overall functionality of the LOM170 according to various embodiments, various structures associated withthe functionality will now be discussed in more detail.

FIG. 6 is a diagram illustrating various functional blocks within theLAN On Motherboard 170 of FIG. 1, according to some embodimentsdisclosed herein. In the embodiment of FIG. 4, the LAN On Motherboard170 includes a host interface 610, a framer 620, adistributor/aggregator 630, and one or more SERDESes 640. The LAN OnMotherboard 170 may also include various other functional blocks notshown in FIG. 6 such as (but not limited to) a management bus interface,a management processor, a bus arbiter, a direct memory access (DMA)engine, auto-negotiation logic, various buffers and memories, and one ormore processors.

As explained above, data is conveyed between the LOM 170 and the hostprocessor 150 over the host data path 210. More specifically, the hostprocessor 150 communicates with a host interface 610 implemented by theLOM 170. To this end, the host interface 610 may include variousconfiguration registers, data registers, mailboxes, shared memorylocations, interrupts, and combinations thereof. As noted above, thehost processor 150 communicates with a particular MAC port 180.Therefore, a MAC port 180 may be implemented as a particular subset ofthese host interface resources.

A framer 620 encapsulates data from the host processor 150 into Ethernetframes, and de-encapsulates data from Ethernet frames received over themidplane 120. Functions implemented by the framer 620 may include, butare not limited to: preamble generation and detection; insertion and/orremoval of source and destination MAC addresses; and Frame CheckSequence (FCS) generation, insertion, and removal.

A distributor/aggregator 630 distributes the Ethernet frames generatedby the framer 620 to the appropriate one of the SERDESes 640. In thereverse direction, the distributor/aggregator 630 collects Ethernetframes received over the midplane 120 and aggregates them into onestream for processing by the framer 620. Each of the SERDESes 640 isassociated with a SERDES lane 220. The embodiment shown in FIG. 6 usestwo SERDES lanes 220, each operating at 25 Gb/s, to achieve a 50 Gb/slink rate. In some embodiments, the host interface 610 and the framer620 are grouped together as MAC circuitry while thedistributor/aggregator 630 and the SERDESes 640 are grouped together asphysical layer (PHY) circuitry. In some embodiments, thedistributor/aggregator 630 resides in Physical Coding Sublayer (PCS)circuitry 650 and the SERDESes 640 reside in Physical Medium Attachmentsublayer (PMA) circuitry 660.

Having introduced the distributor/aggregator 630, frame distributionamong lanes performed by the LOM 170 will now be described in furtherdetail. The IEEE 802.3 standard for 40 Gb/s and 100 Gb/s Ethernetemploys multi-lane distribution (MLD) to distribute data from a singleMAC port across a number of virtual lanes. For a given operating speed,the number of virtual lanes, also referred to as Physical CodingSublayer (PCS) lanes, is determined by the least common multiple (LCM)of the desired range of Physical Medium Dependent (PMD) lanes. In thecase of 100 Gb/s, the desired range of PMD lanes is 1, 2, 4, 5, and 10,which yields an LCM of 20. Thus, the IEEE 802.3 standard for 100 Gb/sEthernet uses MLD striped across 20 virtual lanes. This is sometimesreferred to as MLD20. In the case of 40 Gb/s, the desired range of PMDlanes is 1, 2 and 4, which yields an LCM of 4. Thus, the IEEE 802.3standard for 40 Gb/s Ethernet uses MLD striped across 4 virtual lanes,and sometimes referred to as MLD4.

The MLD striping technique described in the IEEE standard for 100 Gb/sand 40 Gb/s uses a unique alignment marker (AM) for each virtual lane.The AMs are inserted into the striped data stream every 16,000codewords, where every codeword employs 64b/66b encoding. The use of aunique AM for each lane supports three different functions of the MLDstriping technique: lane identification; lane alignment; and bit-levelmultiplexing/demultiplexing. In the case of 100 Gb/s using MLD20, thereare 20 unique AMs, one for each virtual lane. In the case of 40 Gb/susing MLD4, there are 4 unique AMs. In some embodiments of the LAN OnMotherboard 170, the virtual lane alignment markers are identical to theones used for PCS lane 0, 1, 2, and 3 of the IEEE 802.3 standard for 40GBASE-R.

FIG. 7A illustrates another embodiment of the LAN On Motherboard 170,one which utilizes two SERDES lanes. In this embodiment, thedistributor/aggregator 430A uses MLD4 to stripe across four PCS lanes710. Since the number of PCS lanes is greater than the number of SERDESlanes, multiplexers/demultiplexers are used. More specifically, a 2:1bit-level multiplexer/demultiplexer circuit 720 is located in front ofeach of two 25.78 Gb/s backplane SERDESes 640. The two SERDESes 440 thencommunicate the two SERDES lanes 220 across a backplane link 195 (FIG.1).

FIG. 7B illustrates yet another embodiment of the LAN On Motherboard170, one which utilizes four SERDES lanes. In this embodiment, thedistributor/aggregator 630B uses MLD4 to stripe across four PCS lanes710. Since the number of PCS lanes is the same as the number of SERDESlanes, no multiplexer/demultiplexer is used, and the four SERDESes 440communicate the four SERDES lanes 220 across a backplane link 195 (FIG.1).

FIG. 7C illustrates still another embodiment of the LAN On Motherboard170, one which utilizes one SERDES lane. In this embodiment, thedistributor/aggregator 430C uses MLD4 to stripe across four PCS lanes710. Since the number of PCS lanes is greater than the number of SERDESlanes, a multiplexer/demultiplexer is used. More specifically, a 4:1bit-level multiplexer/demultiplexer circuit 730 is located in front of asingle 50 Gb/s backplane SERDES 440, and the SERDES 440 communicates thefour SERDES lanes 220 across a backplane link 195 (FIG. 1). Bit-levelmultiplexers having sizes other than 4:1 and 2:1 can be utilized in asimilar manner.

FIG. 8 is a flowchart illustrating transmit operations of the LAN OnMotherboard 170, according to some embodiments disclosed herein.Alternatively, the flowchart of FIG. 8 may be viewed as implementingvarious steps of a method for conveying Ethernet frames across amidplane, as performed by the LAN On Motherboard 170. At block 810, amedia access control (MAC) interface is provided. The MAC interface isassociated with a port 180 having a 50 Gb/s link rate. Next, at block820, Ethernet frames are generated from data received at the MACinterface. At block 830, the stream of Ethernet frames is encoded intoblocks and PCS lane alignment markers are periodically added to allowthe receive side to deskew the lanes. At block 840, the PCS lanes aredistributed across N SERDES lanes. In some embodiments, M=4 and thedistribution uses the MLD4 striping technique from IEEE 802.3. Next, atblock 850, the distributed Ethernet frames are transmitted on each ofthe SERDES lanes at a 50/N Gb/s rate.

FIG. 9 is a flowchart illustrating receive operations of the LAN OnMotherboard 170, according to some embodiments disclosed herein.Alternatively, the flowchart of FIG. 9 may be viewed as implementingvarious steps of a method for conveying Ethernet frames across amidplane, as performed by the LAN On Motherboard 170. At block 910, amedia access control (MAC) interface is provided. The MAC interface isassociated with a port 180 having a 50 Gb/s link rate. At block 910,Ethernet frames are received on each of N SERDES lanes 220, associatedwith the port 180, at a rate of 50/N Gb/s. At block 920, Ethernet framesare demultiplexed from N SERDES to M PCS lanes. In some embodiments, M=4and the MLD4 striping technique from IEEE 802.3 is used. At block 930,the M PCS lanes are aligned using alignment markers within the blocksthat encode the received Ethernet frames. Once alignment is performed,the alignment markers are removed. At block 940, the aligned frames arebonded or aggregated into a single stream of Ethernet frames. Next, atblock 950, the data from the bonded stream of Ethernet frames isprovided to a 50 Gb/s port of a MAC interface.

The LAN On Motherboard 170 and other various components described hereinmay be embodied in dedicated hardware, software executing ongeneral-purpose hardware, software executing on special-purposehardware, or a combination thereof. If embodied in dedicated hardware,the LOM 170 can be implemented as circuitry and/or a state machine thatemploys any one of or a combination of a number of technologies. Thesetechnologies may include, but are not limited to, discrete logic, aprogrammable logic device, an application specific integrated circuit(ASIC), a field programmable gate array (FPGA), a system on chip (SoC),a system in package (SiP), or any other hardware device having logicgates for implementing various logic functions upon an application ofone or more data signals. Such technologies are generally well known bythose skilled in the art and, consequently, are not described, in detailherein.

When implemented as software, i.e., instructions executing on aprocessor, any logic described herein (including the LOM 170) thatcomprises software or code can be embodied in any non-transitorycomputer-readable medium for use by or in connection with a processor.In this sense, the logic may comprise, for example, statements includinginstructions and declarations that can be fetched from thecomputer-readable medium and executed by a processor. In the context ofthe present disclosure, a “computer-readable medium” can be any mediumthat can contain, store, or maintain the logic or application describedherein for use by or in connection with the processor. Thecomputer-readable medium can comprise any one of many physical mediasuch as, for example, magnetic, optical, or semiconductor media. Morespecific examples of a suitable computer-readable medium would include,but are not limited to, magnetic tapes, magnetic floppy diskettes,magnetic hard drives, memory cards, solid-state drives, USB flashdrives, or optical discs. Also, the computer-readable medium may be arandom access memory (RAM) including, for example, static random accessmemory (SRAM) and dynamic random access memory (DRAM), or magneticrandom access memory (MRAM). In addition, the computer-readable mediummay be a read-only memory (ROM), a programmable read-only memory (PROM),an erasable programmable read-only memory (EPROM), an electricallyerasable programmable read-only memory (EEPROM), or other type of memorydevice.

The diagrams of FIGS. 8 and 9 show the functionality and operation of animplementation of portions of the LOM 170. If embodied in software, eachblock may represent a module, segment, or portion of code that comprisesprogram instructions to implement the specified logical function(s). Theprogram instructions may be embodied in the form of source code thatcomprises human-readable statements written in a programming language ormachine code that comprises instructions recognizable by a processor.The machine code may be converted from the source code, etc. If embodiedin hardware, each block may represent a functional block of specializedhardware, a circuit, or a number of interconnected circuits to implementthe specified logical function(s).

Although the diagrams of FIGS. 8 and 9 show a specific order ofexecution, it is understood that the order of execution may differ fromthat which is depicted. For example, the order of execution of two ormore blocks may be scrambled relative to the order shown. Also, two ormore blocks shown in succession in the diagrams of FIGS. 8 and 9 mayoccur concurrently or with partial concurrence. Further, in someembodiments, one or more of the blocks shown in the diagram of FIGS. 8and 9 may be skipped or omitted. In addition, any number of counters,state variables, semaphores, or messages might be added to the logicalflow described herein, for purposes of enhanced utility, accounting,performance measurement, or providing troubleshooting aids, etc. It isunderstood that all such variations are within the scope of the presentdisclosure. It is understood that the diagrams of FIGS. 8 and 9 merelyprovide an example of the many different types of functionalarrangements that may be employed to implement the operation ofportion(s) of the LOM 170 as described herein. As an alternative, thediagrams of FIGS. 8 and 9 may be viewed as depicting an example of stepsof a method implemented in the LOM 170 according to one or moreembodiments.

It should be emphasized that the above-described embodiments of thepresent disclosure are merely possible examples of implementations setforth for a clear understanding of the principles of the disclosure.Many variations and modifications may be made to the above-describedembodiment(s) without departing substantially from the spirit andprinciples of the disclosure. All such modifications and variations areintended to be included herein within the scope of this disclosure andprotected by the following claims.

The invention claimed is:
 1. A device comprising: circuitry configuredto function as a media access control (MAC) interface associated with aport having an M Gb/s link rate, M being a value greater than or equalto 10; generate Ethernet frames from data received at the MAC interface;distribute the Ethernet frames across a group of serializer/deserializer(SERDES) lanes associated with the port, the group having a size N whichis greater than 1; and transmit the distributed Ethernet frames on eachof the SERDES lanes at an M/N Gb/s rate.
 2. The device of claim 1,wherein the circuitry is configured to: receive Ethernet frames on eachof the SERDES lanes at a M/N Gb/s rate; bond the SERDES lanes into astream of received Ethernet frames; and provide data from the bondedstream of Ethernet frames to the MAC port at an M Gb/s rate.
 3. Thedevice of claim 1, wherein M is
 50. 4. The device of claim 1, wherein Nis 2or
 4. 5. The device of claim 1, wherein the SERDES lanes arearranged to electrically couple to a corresponding plurality of SERDESlanes on a midplane of a blade server chassis.
 6. The device of claim 5,wherein each of the SERDES lanes on the midplane is associated with anelectrical trace group on the midplane.
 7. The device of claim 6,wherein each of electrical trace group includes a transmit differentialpair and a receive differential pair.
 8. The device of claim 1, whereinthe circuitry is configured to distribute the Ethernet frames across theN SERDES lanes using Multi-Lane Distribution across a plurality ofPhysical Coding Sublayer (PCS) lanes.
 9. The device of claim 1, whereinthe device is configured to support another link rate lower than M Gb/sand to autonegotiate between the link rates.
 10. The device of claim 9,wherein the autonegotiation is conducted on a lane 0 of the SERDESlanes.
 11. The device of claim 9, wherein the autonegotiation isconducted in accordance with IEEE 802.3an.
 12. A device comprising: aMedia Access Control (MAC) interface associated with an M Gb/s port andconfigured to generate Ethernet frames from data received at the MACinterface, M being a value greater than or equal to 10; a group ofserializer/deserializer (SERDES) lanes associated with the port, thegroup having a size N which is greater than 1; and a physical layertransceiver (PHY) configured to distribute the Ethernet frames acrossthe group of SERDES lanes, wherein the SERDES lanes are each configuredto transmit distributed Ethernet frames at a M/N Gb/s rate.
 13. Thedevice of claim 12, wherein M is
 50. 14. The device of claim 12, whereinN is 2or
 4. 15. The device of claim 12, wherein the SERDES lanes residein the PHY.
 16. The device of claim 12, wherein at least a first SERDESlane of the group of SERDES lanes is arranged to electrically couple toa corresponding at least first SERDES lane on a midplane of a bladeserver chassis and at least a second SERDES lane of the group of SERDESlanes is arranged to electrically couple to a corresponding at leastsecond SERDES lane on the midplane.
 17. The device of claim 12, whereineach of the SERDES lanes are arranged to electrically couple to acorresponding plurality of SERDES lanes on a midplane of a blade serverchassis.
 18. The device of claim 17, wherein each of the SERDES lanes onthe midplane are associated with an electrical trace group on themidplane.
 19. The device of claim 12, wherein N=2and the PHY is furtherconfigured to distribute the Ethernet frames over 4 virtual lanes, andthe device further comprises a 2:1 bit level multiplexer configured tomultiplex the 4 virtual lanes across the 2 SERDES lanes.
 20. A devicecomprising: circuitry configured to generate Ethernet frames from datareceived at a media access control (MAC) interface associated with aport having a 50 Gb/s link rate; distribute the Ethernet frames across aplurality of serializer/deserializer (SERDES) lanes associated with theport; and transmit the distributed Ethernet frames on each of the SERDESlanes at a 50/N Gb/s rate, where N is the number of the plurality ofSERDES lanes.